Social Choice for Human Computation

Designers of human computation systems often face the need to aggregate noisy information provided by multiple people. While voting is often used for this purpose, the specific voting methods that are employed are typically naı̈ve. The theory of social choice provides powerful tools for exactly these settings. We conduct experiments on Amazon Mechanical Turk which demonstrate empirically that more intricate voting rules, which are known to provide theoretical guarantees under simple models of noisy votes, significantly outperform basic voting rules. Our work has the potential to spark a longterm interaction between human computation and (computational) social choice.